1. Field of the Invention
The present invention is directed to methods of determining measurements using a single image.
2. Description of the Related Art
It is known to enhance television broadcasts of events by providing graphics that are blended with other images or video to convey information to viewers. One system for blending a graphic into other images or video is described in U.S. Pat. No. 6,229,550 entitled, “Blending a Graphic.” As disclosed in that reference, the blended graphic can be used in a wide variety of broadcast scenarios, such as broadcasts of sporting competitions and other events. For example, during the broadcast of a football game, a graphic can be employed to indicate the field location to which an offensive team must advance in order to obtain a first down. Logos and other graphics can also be blended with a video of an event. The graphic may be blended in with the live video feed, or in a replay of a live video feed.
One of the challenges in blending a graphic into a video is determining the position of a point in real world space from a depiction of that point in a two dimensional video image. In conventional systems, this may be done for example using in part the known registered camera position and view in real world space, and linear or nonlinear transformation equations to obtain the 3-dimensional position of a point in world space from a video frame showing that point. A time may be associated with a video frame to provide the time dimension of the point at the given position in world space. The registered camera orientation may be obtained from sensors associated with the camera that provide precise information relating to the pan, tilt, zoom and focal distance of the camera.
In conventional systems, this information alone is still generally insufficient to locate the position of a point in three dimensional space. In particular, what is lost when moving from a 2-dimensional video frame to real world space using a registered camera is the depth, or distance between the registered camera and the point in 3-dimensional space. That is, the ray, or “line of position,” from the camera along which the point lies can be determined. But the actual distance between the camera and the point on the line of position is sometimes unknown.
In order to determine this distance, conventional systems use one of several methods. In a first of such methods, specific locations in the real world space being broadcast may be geometrically modeled. Prior to a broadcast, an operator can measure data from different points in the real world environment to be modeled. Each measured data point may be defined by an x, y and z value relative to an arbitrarily selected origin. For example, in an embodiment relating to football broadcasts, the origin may be selected at a corner of the end zone, the y-axis along the width of the field (e.g., the back of the end zone), the x-axis along the length of the field (e.g., the side line) and the z-axis extending vertically from the field. The operator can measure or use the yard markings on the field to determine the x and y coordinates for most points of interest on the field. A laser plane can be used to measure the corresponding z coordinate. The measured data points may be used to define the geometric model (either using the measured points themselves, or fitting the measured data points to one or more curves that mathematically represent the real world environment of the broadcast).
Using the geometric model of the field, the registered parameters of the camera, and the known position of the camera in the geometric model, a real world position of any point may be determined from a 2-dimension image of that point, provided the point lies on the geometric model. In this instance, the world 3-dimensional location of the point is the intersection of the known line of position through the point with the known geometric model of the environment.
At times, it may be desired to locate the 3-dimensional position of a point that may not lie on the geometric model. For example, a thrown football or baseball is generally not defined on the geometric model of the environment. In this instance, a conventional method used to determine the real world position of a point at a given time is to use two registered cameras, or one registered camera in two different locations. In this method, the line of position from the first and second camera views through the point intersect each other at the point. Using matrix transformation, together with the registered positions of the camera views, the position of the point in 3-dimensional world space may be determined.
None of the above-described methods contemplate determining the position of a point in the world space not included on the geometric model using only a single video image.